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The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 
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5) D Claim(s) is/are allowed. 
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DETAILED ACTION 

This is a first office action in response to application no. 09/825,739 filed 
on 4/3/2001 in which claims 1-16 are presented for examination. 



5 Drawings 

The drawings are objected to because the unlabeled rectangular box(es) 
shown in the drawings should be provided with descriptive text labels--.. 
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in 
reply to the Office action to avoid abandonment of the application. Any amended 

10 replacement drawing sheet should include all of the figures appearing on the 
immediate prior version of the sheet, even if only one figure is being amended. 
The figure or figure number of an amended drawing should not be labeled as 
"amended." If a drawing figure is to be canceled, the appropriate figure must be 
removed from the replacement sheet, and where necessary, the remaining 

15 figures must be renumbered and appropriate changes made to the brief 
description of the several views of the drawings for consistency. Additional 
replacement sheets may be necessary to show the renumbering of the remaining 
figures. The replacement sheet(s) should be labeled "Replacement Sheet" in the 
page header (as per 37 CFR 1 .84(c)) so as not to obstruct any portion of the 

20 drawing figures. If the changes are not accepted by the examiner, the applicant 
will be notified and informed of any required corrective action in the next Office 
action. The objection to the drawings will not be held in abeyance. 



10 
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Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 
U.S.C. 102 that form the basis for the rejections under this section made in this 
Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in a patent granted on an application for patent by another 
filed in the United States before the invention thereof by the applicant for patent, or on an 
international application by another who has fulfilled the requirements of paragraphs (1), (2), 
and (4) of section 371(c) of this title before the invention thereof by the applicant for patent. 



Claims 1-3, 5-11, and 14 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Shaffer et al. (U.S. Pub. No. 2001/0046330). 

Regarding claim 1, Shaffer et al. teaches a method for classifying blocks of data 
1 5 comprising the steps of: capturing a block of non- textual data using a recording device 
(para 25, lines 21-24) for which settings for data-capture attributes are indicative of 
characteristics of non-textual data (para 27, lines 14-23); linking meta-data with the block 
of non- textual data, the meta-data corresponding to at least one data-capture attribute 
during capture by the recording device (para 27, lines 14-23; para 25, lines 32-38); and 
20 performing automated processing to assign description to contents of the block, including 
utilizing the meta-data in determining the description (para 32). 

Regarding claim 2, Shaffer et al. teaches the step of capturing includes recording 
at least one of an image file by an image-capture device (para 25, lines 21-24) and audio 
file by an audio recorder. 
25 Regarding claim 3, Shaffer et al. teaches the step of linking includes obtaining 

exposure information that identifies an exposure setting of the recording device (para 27, 
lines 14-23). 
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Regarding claim 5, Shaffer et al. teaches a step of transmitting (Fig. 4, element 
92) the block of the non-textual data and the meta-data from the recording device to a 
computer for performing the automated processing (Fig. 4, element 94). 

Regarding claim 6, Shaffer et al. teaches the automated processing includes 
5 analyzing the non-textual data and the meta-data to identify content-based information 
and manipulating the content-based information to derive the description (Fig. 3, 
elements 64 and 68; para 31, lines 54-58 and 61-66; para 32, lines 1-11; para 28). 

Regarding claim 7, Shaffer et al. teaches the step of analyzing includes applying 
digital signal processing (DSP) to the non-textual data (para 31, lines 61-64). 
1 0 Regarding claim 8, Shaffer et al. teaches the step of performing the automated 

processing includes assigning a semantic expression to the block of non-textual data for 
use as at least one descriptor for one of organizing the blocks of data (para 32) matching 
a query during a search for the block of non-textual data (para 31, lines 56-58; para 34). 
Regarding claim 9, Shaffer et al. teaches a system for classifying subject data 
1 5 comprising: a recording device for capturing non-textual subject data (para 25, lines 21- 
24) and for recording meta-data, the meta-data being specific to an operational mode of 
the recording device during capturing of the non-textual subject data (para 27, lines 14- 
23); and a processor configured to implement a classification technique utilizing both of 
the non- textual subject data and the meta-data for identifying at least one classifier, the 
20 classifier being representative of an attribute of the subject data (Fig. 3, elements 64 and 
68; para 31, lines 54-58 and 61-66; para 32, lines 1-11; para 28). 

Regarding claim 10, Shaffer et al. teaches the recording device is a digital camera 
(para 25, lines 21-24). 
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Regarding claim 1 1, Shaffer et al. teaches the operational mode of the recording 
device is based on a state as determined by at least one of exposure time, auto gain 
setting, film speed, shutter speed, white balance, aperture/lens index, focusing distance, 
and flash/no flash operation (para 27, lines 1 1-23). 
5 Regarding claim 14, Shaffer et al. teaches a method of categorizing files of non- 

textual data comprising the steps of: establishing an evaluation system for decision 
making, including using automated processing techniques to define a plurality of 
algorithms, the algorithms utilizing both of content-based data and meta-data (para 31, 
lines 54-58 and 61-66; para 28), the content-based data corresponding to content 

1 0 information of a file of the non-textual data and the meta-data corresponding to data- 
capturing settings of a data-capturing device during capture of the file of non-textual data 
(para 27, lines 5-23); capturing a file of non-textual subject data (para 25, lines 21-24); 
and processing the file of non- textual subject data through the evaluation system for 
decision making to selectively identify a plurality of classifiers associated with the file of 

1 5 non- textual subject data (para 32). 



Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for 
all obviousness rejections set forth in this Office action: 

20 (a) A patent may not be obtained though the invention is not identically disclosed or described 

as set forth in section 102 of this title, if the differences between the subject matter sought to 
be patented and the prior art are such that the subject matter as a whole would have been 
obvious at the time the invention was made to a person having ordinary skill in the art to which 
said subject matter pertains. Patentability shall not be negatived by the manner in which the 

25 invention was made. 
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Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Shaffer et al. (U.S. Pub. No. 2001/0046330) in view of Exif v. 2.1 (JEIDA^49- 
1998). 

Regarding claim 4, Shaffer et al. teaches the step of capturing further includes 
5 configuring the block as a file of non-textual data in a digital format and wherein the step 
of linking includes forming a tag to the file (para 27, lines 14-23), including a teaching 
that the metadata may comprise exposure parameters. Shaffer et al does not specifically 
teach the tag is indicative of a plurality of exposure time, automatic gain, film speed, 
shutter speed, white balance, aperture/lens index, focusing index, and flash/no flash 
1 0 operation. 

However, the Exif v. 2.1 standard specifically teaches additional tag attributes 
including: exposure time, aperature, flash, and shutter speed (Exif v. 2.1, JEIDA-49- 
1998, p. 35). It would have been obvious to one of ordinary skill in the art to provide the 
Exif standard because it is a "standard for the image file format used in digital still 

1 5 cameras" that satisfies a desire for "uniformity of the attribute information that can be 
recorded in a file" (Exif v. 2.1, JEIDA-49-1998, pg. "Revision History"). As a result, it 
would have been obvious to one of ordinary skill in the art at the time of the invention to 
provide the Exif standard because it is a "standard for the image file format used in 
digital still cameras" that satisfies a desire for "uniformity of the attribute information 

20 that can be recorded in a file". 
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Claims 12-13 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Shaffer et al. (U.S. Pub. No. 2001/0046330) in view of Spence et al. 
(U.S. Pat. No. 6,018,728). 

Regarding claim 12, Shaffer et al. teaches a "class of algorithms employed for this 
5 grouping process would include image processing algorithms which identify objects and 
feature sets within an image by way of the pixel information" (para 31, lines 61-64). 
Shaffer et al. does not teach the specifics of the image processing algorithms. 

However, Spence et al. teaches a neural network capable of classifying images in 
a hierarchical tree structure (col. 3, lines 59-65; col. 4, lines 6-17; (col. 1, lines 50-52) 
1 0 Furthermore, Spence shows a sequential progression of decision making comprising a 
plurality of classification nodes including algorithms for determining which of a plurality 
of alternative next classification nodes is to be encountered in the sequential progression 
of decision making (Fig. 2). One of ordinary skill in the art would have provided the 
neural network classification method of Spence et al. for the purpose of detecting 
1 5 complex objects by using collections of simple networks in order to produce an overall 
estimate that an object of interest is present (col.4, lines 18-27; col. 2, lines 30-39). As a 
result it would have been obvious to one of ordinary skill in the art at the time of the 
invention to provide the classification method of Spence et al. for the purpose of 
detecting complex objects by using collections of simple networks in order to produce an 
20 overall estimate that an object of interest is present. 

Regarding claim 13, Shaffer et al. teaches a "class of algorithms employed for this 
grouping process would include image processing algorithms which identify objects and 
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feature sets within an image by way of the pixel information" (para 31, lines 61-64). 
Shaffer et al. does not teach the specifics of the image processing algorithms. 
However, Spence et al. teaches a classification technique is a neural network having an 
input stage, an output stage and at least one decision-making stage, the decision-making 
5 stage comprising a plurality of classification nodes, at least some of the classification 
nodes configured to receive a plurality of weighted inputs from other classification nodes 
within the decision-making stage and from the input stage for generating an output as a 
basis for identifying classifiers (col 3, lines 59-65; col. 1, lines 25-31). 

10 Claims 15-16 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Shaffer et al. (U.S. Pub. No. 2001/0046330) in view of Hutcheson et al. 
(U.S. Pat. No. 5,274,714). 

Regarding claim 15, Shaffer et al. teaches grouping images by image content and 
metadata (para 31, lines 46-66). Shaffer et al. does not teach a learning procedure. 

1 5 However, Hutcheson et al. teaches a learning procedure in which the content- 

based data is extracted from each of a plurality of learning images (col. 9, lines 1-41). 
One of ordinary skill in the art would have provided the learning procedure of Hutcheson 
et al., in which content based data is extracted from each of a plurality of images, for 
purpose of training a neural network to recognize a query image (col. 9, lines 1-30). As a 

20 result, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to provide the learning procedure of Hutcheson et al., in which content based 
data is extracted from each of a plurality of images, for the purpose of training a neural 
network to recognize a query image. 
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Although Hutcheson et al. teaches using the content of the image in order to 
recognize images, Hutcheson et al. does not specifically teach grouping images using 
metadata. However, in light of the teaching of Shaffer et al. that images can be grouped 
based on content and metadata (para 31, lines 46-58), it is clear that in the case where 
5 images are grouped based on metadata and further grouped by a neural network based on 
image content, that both content data and metadata (which may comprise operational 
conditions) would be identified for each image. 

Regarding claim 16, Shaffer et al. teaches grouping images by image content and 
metadata (para 31, lines 46-66). Shaffer et al. does not teach a learning method. 
1 0 However, Hutcheson et al. teaches a learning procedure inputting a series of 

representative images typical of those expected to be recognized by the neural network 
and generating a set of parameters which allow the neural network to determine if an 
image belongs in a class of images (col. 9, lines 1-41); the images from which the 
classification parameters are determined constituting a plurality of learning images and 
1 5 the classification parameters derived from the these learning images being learning 

classifiers. One of ordinary skill in the art would have provided the learning procedure of 
Hutcheson et al. using a plurality of learning images, for purpose of training a neural 
network to recognize a query image (col. 9, lines 1-30). As a result, it would have been 
obvious to one of ordinary skill in the art at the time of the invention to provide the 
20 learning procedure of Hutcheson et al. using a plurality of learning images, for purpose of 
training a neural network to recognize a query image. 

Although Hutcheson et al. teaches using the content of the image in order to 
recognize images, Hutcheson et al. does not specifically teach grouping images using 
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metadata. However, in light of the teaching of Shaffer et al. that images can be grouped 
based on content and metadata (para 31, lines 46-58), it is clear that in the case where 
images are grouped based on metadata and further grouped by a neural network based on 
image content, that a learned classification is produced from analysis of image contents 
5 and metadata. 



Conclusion 

Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to Brian Jelinek whose telephone number is 
10 (703) 305-4724. The examiner can normally be reached on M-F 8:00 am - 4:00 
pm. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Andrew Christensen can be reached on (703) 308-9644. 
The fax phone number for the organization where this application or proceeding 

1 5 is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public 
PAIR. Status information for unpublished applications is available through 

20 Private PAIR only. For more information about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll- 
free). 
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